TABLEAU COMPLEXES 

ALLEN KNUTSON, EZRA MILLER, AND ALEXANDER YONG 

Abstract. Let X, Y be finite sets and T a set of functions from X — > Y which we will 
call "tableaux". We define a simplicial complex whose facets, all of the same dimension, 
correspond to these tableaux. Such tableau complexes have many nice properties, and are 
frequently homeomorphic to balls, which we prove using vertex decompositions |BP79|. 

In our motivating example, the facets are labeled by semistandard Young tableaux, and 
the more general interior faces are labeled by Buch's set-valued semistandard tableaux. 
One vertex decomposition of this "Young tableau complex" parallels Lascoux's transition 
formula for vexillary double Grothendieck polynomials |La01 La03|. Consequently, we 
obtain formulae (both old and new) for these polynomials. In particular, we present a 
common generalization of the formulae of Wachs |Wa85| and Buch LBu02il . each of which 
implies the classical tableau formula for Schur polynomials. 
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1. Introduction 



1.1. Statement of results. Let X and Y be two finite sets. We will call functions from X 
to Y tableaux; we think of each tableau f : X ^ Y as a labeling of the points of X by 
elements of Y. Formally, we identify a tableau f with its corresponding set {(x i— > y) : 
f (x) = ij} C X X Y of ordered pairs, whose projection to X is bijective. 

We specify a subset T of "special" tableaux. Also, let E C X x Y be a relation containing 
every f G T. There are obvious minimal and maximal choices of E, namely U T := UfeT ^ 
and X X Y, but it will be convenient to not restrict E. 

Our motivating example is when X is the set of boxes in partition A and Y = {1 , . . . , n}, 
so a tableau f : X ^ Y is a Young tableau of shape A and with entries bounded above by n 
(without any other demands on the labeling), and T is the special subset of semistandard 
Young tableaux. In a moment (Section [L2||, we will describe this case in detail. 

Define the simplicial complex Ae(X ^ Y), which we call a tableau complex, as follows. 
Consider the collection of subsets of E as a simplex under reverse inclusion; thus the 
vertices are the complements (xh^ij) := E\{(x i-^' y]} rather than the elements (x i-^' 
y) G E. We view the faces of this simplex as set-valued tableaux, thought of as relations 
E : X => Y, in which every element x G X is labeled by a set of elements E(x) C Y. A 
set-valued tableau E' is a face of F whenever E' 3 E, meaning that E'(x) 3 E(x) for all x G 
X. (This set-theoretic containment is always intended when we say that one set-valued 
tableau contains another, even when both are being considered as faces of a simplicial 
complex, where containment among faces goes the opposite way.) The tableau complex 
is defined by its facets (maximal faces), which we declare to be the tableaux f G T. (In 
this paper, the terms "function" and "tableau", when unadorned by "set-valued", mean 
single-valued functions in the usual sense.) The face with no vertices, which we call the 
empty face, is the set-valued tableau E. 

Example. Consider the tableau complex in which X = {1,2,3,4} and Y is the English al- 
phabet, and where T consists of all the English words in [dh]e[al][dl]. In detail, E = {(1 i— > 
d], (1 t-^ h), (2 t-^ e), (3 a), (3 i-^ I), (4 i-^ d), (4 t-4 I)}. For simplicity, in the following 
figure, at the vertices we indicate the unused letters of E. For example, the vertex common 



h 




to "hell", "heal", "head" and "held" is (1 m> d) = E \ {(1 ^ d)}. This complex is a 2-ball, 
but if "deld" were a word, it would label the outer face, and this complex would be a 
2-sphere. 

Theorem A. The following hold for an arbitrary tableau complex A = Ae(X Y). 
1. A is pure, meaning that its facets (the tableaux f G T) all have the same dimension. 
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2. The codimension of a face F in A is the number Y.x£xi\^M\ ~ ^ ) of its "extra" values. For 
example, faces of codimension 1 , which we call ridges, are set-valued tableaux taking two 
values for precisely one x G X and taking one value at all other points ofX. 

3. Each ridge is a face of at most two facets. In particular, if a tableau complex is shellable then 
it is homeomorphic to a ball or sphere. 

4. The link of a face in a tableau complex is again a tableau complex. 

All of these results are proved in Section IZTI except the last (the statement about links), 
which is Proposition 12.31 

We shall see that abstractly, a tableau complex is a (multicone on a) top-dimensional 
subcomplex of a join of boundaries of simplices. Tableau complexes can also be character- 
ized, among pure complexes, by the extremal property given toward the end of Section|21 

Although tableau complexes are not generally (shellable) balls or spheres, we can give 
conditions that guarantee this conclusion. The next theorem thus defines the main class 
of tableau complexes of interest in this paper. Except for the claim about interior faces, 
which is Proposition 12.21 it is a simpler-to-state special case of Theorem 12.81 

Theorem B. Let Xbe a poset, and Y totally ordered. Let W be a set of pairs (xi,X2) in X with 
xi < xj. Let T be the set of tableaux f : X ^ Y such that 

• ifxf < X2, then f(xi) < f(x2), and 

• if{x^,xz) G V, then f(xi) < f(x2); 

thus T consists of the order-preserving tableaux from X to Y that are strictly order-preserving on 
the pairs in W. Let E 3 |J T. Then the tableau complex Ae(X ^ Y] is 

1. homeomorphic to a ball or sphere; 

2. vertex-decomposable, as defined in |IBP791, and hence shellable; and 

3. a manifold with (possibly empty) boundary whose interior faces are those set-valued tab- 
leaux F such that every tableau f C F lies in T. 

If Y is taken to be a set of natural numbers, then the tableaux in Theorem |Bl are P- 
partitions [St98|, where X = P. In our context, however, this point of view is misleading 
for a couple of reasons. First, the condition that Y be totally ordered can be relaxed in a 
natural way, as we will see during the proof. Second, P-partitions naturally form a set that 
is infinite and possesses additive structure; both of these properties are unnatural from 
the point of view of tableau complexes. More deeply, P-partitions correspond naturally to 
the basis elements of the Stanley-Reisner ring of a certain simplicial complex (a Grobner 
degeneration of the cone of P-partitions) rather than to the facets. 

We give three formulae for the Fiilbert series of the Stanley-Reisner ring, the third one 
based on an explicit shelling of tableau complexes. For proofs, see Section 01 where the 
statements break the products over v 3 F and v ^ F further into products over x G X. 

Theorem C. Let A = A^{X-^Y) be a tableau complex, and recall that the vertices of A are 
set-valued tableaux (x y) C E. 

1. The Hilbert series, in variables {tv : v is a vertex of A}, equals Ka/ HH ~ tv), where the 
denominator product is over all vertices v of A, and the numerator is the K-polynomial 
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the sum being over all set-valued tableaux F C E such that f C T for some f G T. 

2. If A is homeomorphic to a ball or a sphere, then writing |F| = ^^ex 1^(^)1 ^^'^ Wfor the 
size ofX, the ^.-polynomial can be expressed an alternating sum 

over the set-valued tableaux F C E such that every tableau f C F satisfies f G T. 

3. Assume furthermore the hypotheses of Theorem^ and sei E = |J T. Then there is a shelling 
for A such that the minimal new face when the facet f G T zs added during the shelling is 
an explicitly described set-valued tableau N(f) 3 f. Consequently, 

^^=Lnc -^-^ n ^- 

fGT v2f v3N(f) 

As a result of Theorem lCI31 we get a positive combinatorial rule to compute the h- vector 
(Ho.Hi,... ) of A: ifri(f] |E \ N (f)| - 1, then Hj counts the number of f G Twithri(f) = j. 



1.2. Young tableau complexes. We now describe the prototypical example of a tableau 
complex, and an application to computing vexillary Grothendieck polynomials, or equiv- 
alently, Hilbert series formulae for vexillary determtnantal varieties. 

Let A C be an English partition, or equivalently, a Young shape with its origin at its 
upper-left corner. A set-valued Young tableau |Bu02| is a filling of the boxes of A, each 
with a nonempty finite set of natural numbers. The set in each box is typically expressed 
as a strictly increasing list. When the set in every box is a singleton, what results is an 
(ordinary) Young tableaux. If |t| denotes the number of entries in a set-valued tableau t, 
and |A| is the number of boxes in the partition, then |Tj > |A|. Moreover, |t| = |A| only for 
tableaux. (Tableaux are assumed ordinary unless the term "set-valued" is written.) 

A set-valued tableau t is called semistandard if for every pair bi , of boxes of t, 

• each entry of bi is weakly less than each entry of bi whenever bi lies left of bi, and 

• each entry of bi is strictly less than each entry of b2 whenever bi lies above bi. 

One can speak of one set-valued tableau containing another (of the same shape A) if for 
each box of A, the set of numbers in one set-valued tableau contains the corresponding 
set in the other. In these terms, for t to be semistandard, one needs that every tableau 
contained in t is semistandard in the usual sense. More generally, we define a set-valued 
tableau to be limit semistandard if some tableau it contains is semistandard. For example, 
the first of the following set-valued tableaux is semistandard, the second is limit semis- 
tandard, and the third is neither: 



1,3 


3 


3,6 




1,4 


2,3 


3 


4,5 
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9 
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4,5 
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8 






4 





2,4 


3,4 


3 


2,3 


3,5 


4 


6 





Fiereafter, we will not bother to write the commas in our examples; no confusion will 
result because we only use numbers that are at most 9. 

The union of two set-valued tableaux of the same shape A simply assigns to each box of 
A the union of the two sets associated to it. Moreover, if either set-valued tableau is limit 
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Figure 1 . A Young tableau complex. At left is the empty-face tableau. 



semistandard, so is the union. The intersection is not always defined, however, because 
of the requirement that every box of A be nonempty. 

In addition to the partition A, fix a maximum entry value n G ={1,2,3,...}. Define 
the empty-face tableau Ea^u associated to A and n as the union of all the semistandard 
tableaux with shape A and all entries at most n. 

Consider the partition A as a poset in which (i, j) < (!', j') whenever i < i' and ] <]'; 
thus each box is less than the boxes southeast of it. Writing [n] = {1 , . . . , n}, we get a 
tableau complex ^^^'^ W )/ in the sense of Theorem |Bl take ^ to be the set of pairs 
(upper box, lower box] in which one box sits atop another in A, so T is the set of semi- 
standard Young tableaux on A with maximum value n. Observing that n('^~'^ 
depends only on A and n, we denote this Young tableau complex by A(A, n]. See Figure |T] 
for an example. The special case of Theorem|Blfor Young tableau complexes is as follows. 

Corollary. The Young tableau complex A(A, n] zs homeomorphic to a shellable hall or sphere, and 
its interior faces are labeled by Buch's semistandard set-valued Young tableaux |Bu02l- 

Example 1.1. Let A = (2, 1 ) and n = 3. Then Aa,3 is a 3-dimensional ball. It has one interior 
vertex, missing the 2 in the upper left box. We draw the boundary 2-sphere in Figure |2| 
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Figure 2. A triangulated 2-sphere of properly limit semistandard tableaux. 
The two edges with arrows meet around the back side of the sphere. 
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When vertex-decomposing a Young tableau complex — that is, writing it as the union 
of the star and the deletion of a single vertex — the two subcomplexes are flagged Young 
tableau complexes, in which a vector n bounds the sizes of the entries in the rows of A. 
This suggests that we ought to work in that level of generality; see Figure 




Figure 3. The empty-face tableau and simplicial complex for A — (2, 1) 
and n = (2, 3). 

Flagged Young tableaux are used to compute the Schubert polynomials for vexillary 
permutations |Wa85|, and one choice of vertex decomposition parallels the "transition 
formula" for their double Grothendieck polynomials |La01, La03|. Hence we are able to 
give set-valued-tableaux-based formulae for double Grothendieck polynomials of vexil- 
lary permutations, via Theorem |Cl The second formula in the following corollary, which 
appeared already in |KMY05|, is a common generalization of Buch's and Wachs's formu- 
lae, each of which specializes to the usual tableau formula for Schur polynomials. The 
other two parts give new formulae for these polynomials. See Section|5lfor proofs. 

Corollary. Let tt G Snbe a vexillary permutation with associated partition A and flagging n. 
Each of the following is a formula for the double Grothendieck polynomial (Snlx, y). 

1. As a sum over the set LSVT[A, n) of limit semistandard tableau associated to (A, n), 

a5„(x,y)= n n c "''^yiij(b)) n "^^yhim^ 

reLSVT(X,n) bGA iGT(b) hGE^^fi: (b)\T(b) 

where Ex.fL = [j^ is the union of all semistandard tableaux t g SSYT{X,fi), and j(b) ~ 
c(b) — r(b) is the difference of the row and column indices of the box bGA. 

2. As a sum over the set SVT{\, n) of semistandard set-valued tableau associated to (A, ft), 

reSVT(\,fi) beA iGT(b) 
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3. As a sum over the set SSYT{'\, n) of semistandard Young tableaux of shape X flagged by n, 

^7r(x,y)= Y. n n -''^^i^+jib)) n "^^yhim' 

reSSYT{\ri) beA iGT(b) HeE^.^ (b)\NT (b) 

where N^- is the tableau obtained by adding to each box b all numbers in EA,n(b) either 
smaller than the entry o/T(b), or larger than that entry provided that replacing the entry 
with the larger number would not give a tableau in SSYT{X, n). 

The second of these formulae for vexillary double Grothendieck polynomials was based 
on the algebraic geometry of matrix Schubert varieties |KMY05 1. It was that geometry that 
first motivated us to fit Young tableaux into a simplicial complex. 

2. Properties of tableau complexes 

2.1. Generalities and boundary faces. Recall that the vertices of a tableau complex con- 
sist of the complements (x m> y ) of single elements of E. 

Proposition 2.1. Let A = Ae(X ^Y)be a tableau complex, and assume that (x ^— > y ] G E. 

1. A is pure, and its facets are labeled by the tableaux in T. 

2. Writing |E| = ^^gx codimension of a face E in Ae(X Y) equals |E| — |X|. 

3. Each ridge is contained in at most two facets. In particular, if A-^{X Y) is shellable then 
it is homeomorphic to a ball or sphere. 

4. [xt^ y) is a cone vertex (meaning it lies in every^acet) if and only z/f(x) ^ y for every 
f G T. Jn particular, E = |J T exactly when Ae(X — > Y) has no cone vertices. 

5. (x -y ) is a phantom vertex, meaning (x y ) ^ A, precisely iff{x] = yfor all f G T. 

Proof. Only statement |3l is not immediate from the definitions. A ridge is a set-valued 
tableau taking one extra value. By the pigeonhole principle, since every x G X gets at 
least one y G Y, there exists exactly one x with two values from Y, all others being 1 . Such 
a set-valued tableau can contain at most two tableaux from T. The statement about being 
a ball or sphere now follows from IBLSWZ99t Proposit ion 4.7.22]. □ 

Cone vertices are in some sense uninteresting: a simplicial complex can be canonically 
reconstructed from its set of cone vertices and its core, which is the subcomplex with the 
cone vertices removed. In particular, the whole complex is a ball or sphere if and only 
if its core is a ball or sphere. It is convenient for inductive purposes not to assume that 
E = IJ T, although we will generally assume E = |J T in examples. 

Proposition 2.2. Assume that Ae(X-'-)Y) is homeomorphic to a ball or sphere. A face E of 
Ae(X-'->Y) lies on the boundary o/Ae(X-'->Y) if and only if there exists a tableau g : X —> Y 
such that g C E but g ^ T. 

Proof. By definition, a face of a simplicial ball lies in the boundary if and only if it is a 
face of a boundary ridge. A ridge itself lies in the boundary if and only if it is a face of 
precisely one facet. Of the two tableaux contained in a given ridge, at least one must lie 
in T, because a ridge is a face of Ae(X —> Y). Hence a ridge is a boundary face if and only 
if the unique other tableau it contains does not lie in T. 

Now let E be an arbitrary face of Ae(X ^ Y). If every function f C E lies in T, then every 
ridge with E as a face is a union of two tableaux from T, so E is interior. On the other hand. 
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suppose that g C F for some tableau g ^ T, and let f e T be a facet having F as a face, so 
f C F. Then f U g C F is a set-valued tableau that has F as a face. Some of the elements 
X e X are assigned two distinct Y-values by f U g. If deleting the value f (x) from (f U g) (x) 
yields a face G of Ae(X ^ Y), then induction on the codimension implies that G lies on the 
boundary, and hence F 2 G does, as well. If no such x exists, so deleting the value f (x) 
from f U g always results in a set-valued tableau that is not a face of Ah(X ^ Y), then f is 
the unique tableau in T with f U g as a face; thus f U g is a face of only one facet (namely f ), 
and hence f U g is a boundary face with F as a subface. □ 

2.2. Safe vertices in tableau complexes. Given a simplicial complex A with a vertex v, 
define the star and deletion of v to be 

staivA = {CeA:CUveA} and del^A = {C e A : v ^ C}. 

Then A = staivA U delvA. The star has an obvious cone vertex, namely v itself, and its 
deletion from the star is called the link of v in A. More generally, the link of a face C in a 
simplicial complex A is defined as 

linkcA = {D e A : D n C = 0, D U C e A}. 

By convention, the vertex set of this link does not include the (now phantom) vertices of C. 

Proposition 2.3. Let F be a face of A — Ae(X Y). Let Tunk = {f G T : f C F} &e f/ze set of facets 
of A having V asa subface. Then the link of? in A is isomorphic to Af(X Y). 

Proof It follows from the definitions that the faces of both link^A and Af(X Y) are the 
set-valued tableaux contained in F and containing a tableau from T. □ 

Proposition 2.4. Let Ae{X-^Y) be a tableau complex. Let Tgtar = {f e T : f(x) y}. Then 
star(^y)AE(X^Y) = Ae(X^Y). 

Proof Since Ae(X-^Y) is pure, the star of (x is the union of the (closures of) facets 
that have (x y] as a vertex. These facets are exactly the tableaux f e Tgtar- □ 

Call a vertex (x y) of Ae(X —> Y) safe if for every f G T, changing the label on x from 
f (x) to y yields a tableau that is again in T. While the star of a vertex in a pure complex is 
always pure, the deletion might not be. 

Proposition 2.5. The deletion del(xH^tj]A of the vertex (x ^ y)from the simplicial complex A = 
Ae(X ^ Y) is pure if and only if either (x tj ) is a cone vertex or (x -y) is safe. 

Proof. If (x y ) is a cone vertex then A is the cone over del(xh^y)A. A simplicial complex 
is pure if and only if the cone over it is, so we assume that (x m> ij ) is not a cone vertex. 

Given a set- valued tableau F, let del(x^v^y)F denote the set-valued tableau that sends a ^ 
F(a) for a X and sends x h- > F(x) U{y}. In particular, del(xH4y)F = F if and only if y G F(x). 
The definitions imply that del(xh^y)A consists of the set-valued tableaux del(xh4y)F for F g A, 
and the facets of dcl(x^y)F have the form dcl(xH^y)f for tableaux f G T. Since (x m> y) is not 
a cone vertex, at least one facet of A sends x to y. Thus the deletion is pure if and only if, 
for all tableaux f G T, the set-valued tableau de\(^y)i contains a tableau g G T satisfying 
g(x) = y. The desired result follows because when f (x) does not already equal y, the only 
possibility for g is obtained by changing f (x) ioy. □ 
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Corollary 2.6. Let [x\^ y] be a safe vertex of the tableau complex A = Ae(X— > Y). If Jdd = 
{f G T:f(x) = yl then de\^^y) A = A-e{X^\). □ 

2.3. Tableau complexes on posets. At this point we make some additional assumptions 
to guarantee a ready supply of safe vertices. The following theorem is stated much more 
generally than our motivating examples require; we hope that this Bourbakiesque level 
of generality helps to indicate which assumptions are leading to which conclusions. 

The key to our geometric conclusions (shellable ball or sphere) is the notion of vertex- 
decomposable simplicial complex in the sense of pP79|. By definition, every simplex is 
vertex-decomposable, and an arbitrary simplicial complex is vertex-decomposable if and 
only if it is pure and has a vertex whose deletion and link are both vertex-decomposable. 

Lemma 2.7 (IB P79I '). A simplicial complex A is shellable if both the deletion delvA and the star 
staivA of a vertex v are shellable. Hence all vertex-decomposable simplicial complexes are shellable. 

Proof. (|BP79|) Construct a shelling of A by concatenating shellings of the deletion and 
star of V (in that order), the latter being the cone over a shelling of the link. □ 

Theorem 2.8. Let X and Y be finite partially ordered sets. For each x G X, let Y^ be a totally 
ordered subset ofY. Fix a set W of pairs (xi,X2) from X such that xi < xz. Let T be the set of 
tableaux f : X ^ Y such that 

• f (x) G Yx/or all x G X; 

• f is weakly order-preserving, i.e. xi < xz implies that i[x^) < f(x2); and 

• if (xi , xi) G ^ then i[x^ ) < f (xi). 

Let E C X X Y contain |JT. Then Ae(X^Y) is homeomorphic to a ball or sphere, and it is 
vertex-decomposable. 

Proof. We need only prove vertex-decomposability, for then the ball or sphere conclusion 
is a consequence of Proposition 12 . 1I3| and Lemma lZTI To demonstrate vertex-decomposa- 
bility, we need only find, for each tableau complex satisfying the hypotheses of the theo- 
rem, a vertex whose deletion and link both satisfy the hypotheses. 

Suppose that Ae(X^ Y) has a cone vertex (x m> -y). Viewing (x y) as a subset of 
X X Y, we find that (x m> y) already contains IJT by Proposition 12.1141 so A(xh^y)(X^ Y) 
satisfies the conditions of the theorem. This simplicial complex is the link by Proposi- 
tion |23l and it equals the deletion because (x y ) is a cone vertex. 

Now assume that Ae(X ^ Y) has no cone vertices. If all of the vertices of Ae(X ^ Y) 
are phantom, then there is only one facet and we are done. Otherwise, there exists a 
non-phantom vertex (ra y ). Choose one with maximal possible ra, and let ym be the 
maximum element of Ym.. Since there are no cone points, the values of all tableaux in T 
are fixed at elements x > ra: for each x > ra and all f G T there is some yx ^ Y such 
that f (x) = yx. Therefore, as (m m> ym) is itself not a cone vertex, we get ym < yx for all 
X > ra, and yra < if (ra, x) G V. It follows that the vertex (ra ym.) is safe: we can 
safely change the label on ra from f (ra) to ym to get another tableau satisfying the three 
conditions to be in T because y m. > f (m) for all f G T. 

(We used that Ym has a maximum element for this, but not that it is totally ordered.) 

Most of the work has now been done in Section if Tgtar = {f £ T : f(ra] ym} 
and Tdei = {f G T : f(m) = y^}, then the star and deletion of (ra ^ ym) are Ae(X^^ Y) 
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and Ae(X Y), respectively, by Proposition 12.41 and Corollary 12 ■6[ The star and deletion 
satisfy the three conditions from the statement of the theorem, with the same X, Y, and W, 
but with Ym, changed either to Yra \ {Vm} or else to {ym}, respectively Given that the star 
satisfies the hypotheses of the theorem, arguing as in the second paragraph of the proof 
shows that the link does, as well. 

(To work inductively, we need Y^, \ {ym) to again have a maximum element; this is why 
we required Yra to be totally ordered. In addition, our new choice of ra for the link must 
have a maximum element in its Ym) this is why we need every Y^ to have a maximum.) □ 

Before this theorem, we never needed to compare f (xi ) and f (X2) for xi ^ X2; in some 
sense, it would have been more natural for the tableaux to take values in separate sets 
Yx. Now that we used a partial order on Y to define our set T of tableaux, we have finally 
made such comparisons. 

Example 2.9. More generally than in Section 1.2, let X be the set of boxes in a skew-shape 
A/(j., and each Yx = Y = {1 , . . . , n}. Partially order X by asking that each box is less than 
the boxes southeast of it. Let W be the set of pairs {(upper box, lower box)} where one 
box is atop another. Then T is the set of semistandard Young tableaux of shape X/\x with 
maximum value n, and Ay^^^ f (X ^ Y) is the Young skew tableau complex. 

The faces of this complex are labeled with set-valued Young skew tableaux, which were 
also introduced in |Bu02|. Buch's definition of "semistandard" set-valued Young tableaux 
exactly matches our criterion. Proposition 12.21 for a face to be interior. 

(In fact the W machinery was unnecessary to model semistandardness; we could just 
take ¥ = 0, subtract r — 1 from the values in the rth row, and adjust the sets Yx to get a set 
combinatorially equivalent to semistandard Young tableaux. But the formulation with W 
is clearer, more general, and no more difficult.) 

Example 2.10. Let X be a poset, W = 0, and Y = {0, 1}. Then the tableaux correspond to 
partitioning X into a lower and an upper order ideal (the and 1 parts), or equivalently to 
antichains in X (the maximum elements labeled 0). By Theorem 12.81 this tableau complex 
is homeomorphic to a ball (or sphere, if X is totally unordered). 

Remark 2.11. Other classes of vertex-decomposable complexes include the greedoid com- 
plexes |BKL85 1 and subword complexes I KM03I ; see IKM0 3 , Remark 2.6] for an extended 
discussion. Tableau complexes are different from each of these. For example, the Young 
tableau complex for the vertical domino with entries at most 5 is not a greedoid complex if 
the ground set is taken to be the vertex set. To show the difference between subword and 
tableau complexes, consider the Young tableau complex for the 2x2 square shape with 
entries at most 3; it has dimension 3 and eight vertices, none of which is a cone vertex. On 
the other hand, deleting all cone points from the subword complex in |QCMY05 1 having the 
same tableaux for facets yields a simplicial complex of dimension 2 with seven vertices. 

It is worth noting that the phrase "ball or sphere" essentially always really means 
"ball". To get a sphere, there must be no cone vertices, so E = |J T. But even then, 
every ridge lies in two facets, so every vertex must be safe; in other words, the possible 
T- tableau values at each x G X are independent. We spell this out further in Section |3l For 
now, here is a characterization of the interior faces, which includes all of the faces in the 
case of a sphere. As a matter of notation, if Yi and Y2 are two subsets of a poset Y, write 
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Yi < Y2 if <y2 for all iji G Yi and yz G Yz- Similarly, write Yi < Yz if strict inequality 
holds. The following is an immediate consequence of the definitions and Proposition 12.21 

Corollary 2.12. Assume the notation and hypotheses of Theorem \2.8\ A face F is interior to 
Ae(X ^ Y] if and only if 

• < ^{'X-z) whenever x^ < xz; and 

• F(xi) < F(x2) whenever (xi < X2] G W. 

2.4. Shelling poset tableau complexes. The next theorem will help us describe the h- 
vectors and Hilbert series of poset tableau complexes and their Stanley-Reisner rings. 

Theorem 2.13. Assume the notation and hypotheses ofTheorem \2.8\ and choose a linear extension 
e of the partial ordering on X. Lexicographically order T by comparing f 1 , f2 G T the t-largest 
element ra G X where they differ. Placing the one with the larger label on m first yields a total 
order on the facets 0/ Ae(X Y) that is a shelling. 

Proof. Remember that f i (ra) and f2(ra) are comparable, since Ym, is totally ordered. There- 
fore the procedure in the statement of the theorem yields a total order on the facets. We 
will show that this total order is the shelling produced by applying Lemma ITTI recursively 
as in the proof of Theorem 12.81 At each stage in that proof, we either vertex decompose 
at a cone vertex or we choose a maximal element ra G X among those supporting non- 
phantom vertices (ra 1^ y ). Vertex decomposing at a cone vertex does not alter the set T of 
facet tableaux, so it does not matter in which order we delete cone vertices. Only the order 
in which we choose the maximal elements ra matters. Use the e-order: since Lemma IZTl 
puts the deletion (f(ra) = first before the star (f(ra) 7^ y^, which is equivalent to 
f (ra] < Urn because y m is maximum in Yra), the resulting shelling is as desired. □ 

3. Characterizations of tableau complexes 

Let A be a pure simplicial complex on a vertex set V. Declare that W C V is a pure 
factor of A if the number |f fl W| of vertices in the intersection of f with W is the same for 
all facets f G A. For example, a singleton {v} is a pure factor if and only if v is a phantom 
or cone vertex, with |f fl {v}| = or |f fl {v}| = 1, respectively. If W is a pure factor, then its 
complement V \ W is a pure factor, too. For any set W C V of vertices, write A|w for the 
full subcomplex delv\wA supported on W. 

Proposition 3.1. Let Abe a pure simplicial complex on the vertex set V, and suppose that V = 
Vi U ■ ■ ■ U Vk zs partitioned into a disjoint union of pure factors Vi , . . . , V^. Then A\v. is pure for 
each 1 < i < k, and A is a top-dimensional subcomplex of their join 

{Fl U ■ • • U Fk : Fi is a face 0/ Alv^ for each 1 < i < k}. 

Proof. Since A is pure, it follows from the definitions that Ajvt is pure. On the other hand, 
it also follows by definition that a subset f C Vi U ■ ■ • U Vk is a facet of the join of the 
complexes Alv^ if and only if f fl Vt is a facet of the individual complex A| for each i. Since 
the vertex set V is the disjoint union Vi U ■ ■ ■ U V^, every facet of A has this property. □ 

Lemma 3.2. Let Abe a tableau complex A\jj[X Y). For each x G X, define the subset Vx = 
{(x ^y] : (x 1-^ y ) G \Jl]ofthe vertex set of A. Then 

1. the subsets {Vx : x G X} partition the vertex set of A; 
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2. each subset Vx is a -pure factor; in fact, |f fi Vx| = |Vxl — 1 for every facet tableau f; and 

3. each induced complex Ajy^ is the boundary of the simplex on Vx. 

Proof. The first two numbered claims are immediate from the definitions. For the third, 
it follows from the second that A|v^ is a union of some subset of the facets (each of size 
|Vx| — 1) in the boundary of the simplex on Vx. Each facet of A|v^ avoids using some 
(unique) vertex of Vx. If any vertex of Vx does not occur this way, then it lies in every 
facet of Alv^; in other words, it is a cone vertex of A|v^ . Since each facet of A survives after 
deleting V \ Vx to give a facet of A|v^, we conclude that A has a cone vertex, contradicting 
Proposition 12 . 1 14l Therefore every face of size |Vxl — 1 occurs in A|v^. □ 

Theorem 3.3. A pure complex is (isomorphic to) a tableau complex Ay t(X ^ Y) if and only if it 
can be expressed as a top-dimensional subcomplex of a join of boundaries of simplices. 

Proof That AyT(XAY) can be expressed in the desired manner is an immediate conse- 
quence of Proposition 13.11 and Lemma 13.21 Now suppose that that A is a pure complex 
expressible as a top-dimensional subcomplex of the join of boundaries of simplices with 
vertex sets Vi , . . . , V^. Then the vertex set V of A is the disjoint union Vi U ■ ■ ■ U V^. Let 
X = {1 , . . . , k} and set Y = V. Each facet f of A defines a function X ^ Y taking i to the 
element Vi \ f . Using these as the set T of tableaux, we find that A = Ay t(X A Y). □ 

Remark 3.4. In particular, if a tableau complex has no boundary ridges (ridges contained 
in just one facet), then it is the join of a bunch of boundaries of simplices, and in particular 
it is a sphere. This, plus Proposition |Z21 gives another proof of Proposition 12.21 

Let the codimension of a pure complex A be the number of vertices outside any facet. 
The only way for the codimension to equal is if V consists only of cone vertices (so V is 
a simplex). If V breaks up as a union Vi U ■ ■ ■ U Vi< of pure factors, then the codimension 
of A is the sum of the codimensions of the full subcomplexes supported on the Vi. 

This suggests a characterization of tableau complexes by the following extremal prop- 
erty. Given a pure complex A with the cone and phantom vertices deleted, look for a pure 
factor W. Splitting into W and V \ W, the codimension of each full subcomplex can be no 
larger than that of A. By the theorem, A is a tableau complex if and only if we can split 
enough to whittle the codimensions of all of the full subcomplexes down to 1 . 

The situation is somewhat dual to order complexes of ranked posets. If P is a ranked 
poset, its order complex has vertex set P, and Q C P defines a face if and only if Q is 
totally ordered. If Pr denotes the set of elements with a given rank r, then the induced 
complex on Pr is pure of dimension 0, rather than codimension 1 like a tableau complex. 
(If it seems unsatisfying for "codimension 1 " to be dual to "dimension 0", then consider 
the latter more honestly as "affine-dimension 1 ".) Very few simplicial complexes are both 
order complexes and tableau complexes; we leave their characterization as an exercise for 
the reader. 

Remark 3.5. Tableau complexes bear superficial similarities to matroid complexes. A sim- 
plicial complex is a matroid if and only if every subcomplex induced on a subset of the 
vertex set is pure. Theorem 13.31 says that a simplicial complex is a tableau complex if and 
only if the vertex set can be partitioned into subsets that are pure factors of codimension 1. 
In reality, there are matroid complexes that are not tableau complexes, and conversely. For 
example, we have already seen in Remark |2 . 1 1 1 that tableau complexes can fail to be gree- 
doid complexes, of which matroid complexes are special cases. For an example the other 
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way, the matroid for the complete graph K4 on four vertices is the union of three segments 
joined at a point. If this were a tableau complex, then so would be the result of deleting 
the cone point, by Proposition I2.3[ But a set of three points is not a tableau complex by 
Remark 13.41 and hence neither is the matroid for K4. 

4. HiLBERT SERIES AND K-POLYNOMIALS 

In this section we collect some formulae for the Hilbert series of the Stanley-Reisner 
rings of tableau complexes. Our main reference for generalities on Betti numbers, Hilbert 
series, and K-polynomials (which are numerators of Hilbert series) is IMS04I . For no- 
tation, let k be a field, and set S = lk[V], the polynomial ring in variables v G V in- 
dexed by a finite set V. This is the ambient ring for objects like the Stanley-Reisner ideal 
Ia = (EIveF^ • ^ ^ ^^^^ ^) ^ simplicial complex A with vertex set V, and the 

Stanley-Reisner ring S/Ia- We shall use the alphabet t = {tv : v G V} for finely graded 
Hilbert series and K-polynomials. When A is a tableau complex A£(X ^ Y), recall that V 
is the set {{x y) : [x y) E oi complements of single elements of E. 

Proposition 4.1. The K-polynomial associated to the tableau complex A = Ae(X ^ Y) is 

F xeX \yeE(x)\F(x) y6F{x) / 

the sum being over all set-valued tableaux F C E each containing some tableau f g T. 

Proof. This formula is I MS04i Theorem 1.13] applied to tableau complexes, since the con- 
ditiony G E(x)\F(x) means that (x y) is a vertex of F and the condition y G F(x] means 
that (x y ) is not a vertex of F. □ 

Our second formula uses the ball-or-sphere hypothesis; it therefore holds for (among 
other things) all poset tableau complexes. It will be simpler to prove the formula for 
general balls and spheres first. 

Proposition 4.2. If A is a simplicial ball or sphere with vertex set V, then 

F vgV\F 

where the sum is over all interior faces of A. (All faces are interior if A is a sphere.) 

Proof. Consider the Alexander dual ideal I^ = (IIvgvxf^ : F is a face of A), and start by 
calculating the K-polynomial of I^. The coefficient on the monomial IlvevXF in K(I^; t) is 
the alternating sum of the Betti numbers of I^ in degree HvevvF^ |MS04, Proposition 8.23]. 
By Hochster's formula llMS04i Corollary 1.40], the i*'^ such Betti number equals the di- 
mension dinik Hi_i (linkpA; k) of the reduced homology of the link of F in A, and it comes 
with a sign (—1 If F is a boundary face, then the link of F is contractible; but if F is 
an interior face, then the link of F is a sphere of dimension codim (F) — 1. Therefore 
K(I^;t) = ^p(— 1 HveYXF^w where the sum is over all interior faces F of A. The 

Alexander inversion formula |MS04, Theorem 5.14] now implies the desired result. □ 
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Theorem 4.3. If the tableau complex A = Ae(X — > Y) zs homeomorphic to a ball or sphere, then 

K(S/lA;t)=^(-1)i^Hx|-Q Yl (1-t(,^v)). 

F xeXyeF{x) 
the sum being over all set-valued tableaux F C E such that every tableau f C F lies in T. 

Proof. The factor (—1)11" 1^ is the codimension of F. The condition y G F(x) means that 
[x^ y) lies in the vertex set of A but not F. The sum is over all interior faces by Proposi- 
tion l2.2l Therefore the result is simply Proposition l4.2l for tableau complexes. □ 

A shelling of a pure d-dimensional simplicial complex A is an ordering of the facets 
Fi , . . . , Fic such that F^ fl (Fi U ■ ■ • U Fi_i ) has pure dimension d — 1 for each 1 < i < k. This 
guarantees that for each i, there is a unique minimal new face Ni G A that is a face of Ft 
but not of Fi , . . . , Ft 1 . By convention, N i is the empty face. 

Lemma 4.4. Given a shelling of a simplicial complex A with new faces N i , . . . , Nic as above, 

k 

1=1 v^Ft vgNi 

Proof. Use induction on the number k of facets of A. □ 
The Z-graded coarsening of the Hilbert series to one variable t gives 

When A is shellable, the h-vector (ho, hi , . . . , h^) consists of nonnegative integers. More- 
over, the shelling gives a manifestly positive way to compute these numbers: hj counts 
the number of dimension j faces among N i , . . . , N^. 

Theorem 4.5. Resume the notation and hypotheses of Theorem 12. g[ and sei E = |J T. Given a 
tableau f G T, define Uf as the set of elements y G Y such that f (x) < y and moving the label on x 
from f (x) up to y yields a tableau in T. Then 

K(S/lA;t] = ^ f (1 — t(xf^f(x))) Y\ t(xh4y)j- 

fGT xex ^ yeUf (x) ^ 

Finally, 2/ri(f) = — |X| + ^^ex l^f (^)l' ^^^^ ^'s number of tableaux fGT withr\[-f] = j. 

Proof. The proof will be done once we produce a shelling of A for which Nf = E(x) \ Uf (x) 
is the minimal new face at the stage when we add the facet f . Pick a linear extension e 
of X and take the shelling order of the facets f i , fi, . . . of A as in Theorem l2.13l For f := ft 
we show that Nf is the minimal new face of f. 

First, Nf is a set-valued tableau in A that is a face of f, since it contains f. Second, 
to see that Nf is not a face of any previous facet, we must show that f does not contain 
fi, . . . , fi 1. Note that by construction, any other g G T contained in Nf must assign to 
each X G X either f (x) or some y < f (x). Such a facet tableau g must appear later than f in 
the facet ordering. The maximality of Nf C E containing f and not containing f i , . . . , f| i 
is also clear from the construction. Hence Nf is the minimal new face of f, as claimed. For 
the K-polynomial formula, apply Lemma 231 D 
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Example 4.6. For the tableau complex in Figure |3l (after Example II. listing the facets in 
the order 
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yields the shelling in the proof of Theorem l4.5l For the first of these tableaux, all of the sets 
U(x) are singletons: there is no way to increase the number in any box while respecting 
the flagging, which requires that the entries in the top row are at most 2 and the lower 
entry is at most 3. For the second tableau above, U(x) ={1,2} for the upper-left comer x, 
because moving the 1 up to a 2 keeps the tableau semistandard. Similarly, all of the sets 
U(x) for the third and fourth tableaux are singletons except for upper-right box and the 
bottom box, respectively, whose sets U(x) are {1 , 2} and {2, 3}. In the last tableau above, 
only the two lower-right corners have non-singleton sets U(x), and these are {1 ,2} and 
{2, 3}. For the above five tableau, the function r\ at the end of Theorem I4.5l takes the values 
0, 1, 1, 1, and 2, respectively. Thus the simplicial complex in Figure |21has h- vector (1,3,1). 



Our final K-polynomial formula in this section will arise again after Corollary 15.31 
Proposition 4.7. If [x y) is a safe vertex of A = Ae(X Y), then 

K(S/Ia; t) = t(xH<>y)K(S/Idei;t) + (1 — t(xH^y))K(S/Istar;t), 

where Idci and Istar the Stanley-Reisner ideals for the deletion tableau complex Ae(X^^Y) 
and the star tableau complex Ae(X Y]from Propositions \2.4\ and \2.5\ respectively. 

Proof. Any vertex decomposition A = delvA U staivA gives an inductive formula 

K(S/lA;t) = t,K(S/IdeU.A;t) + (1 - tJK(S/Istar.A;t] 

for the K-polynomial. □ 



5. Applications to vexillary double Grothendieck polynomials 

In this section, we apply Section ID to obtain formulae for double Grothendieck poly- 
nomials for vexillary permutations. This gives formulae for the Hilbert series and K- 
polynomials of vexillary matrix Schubert varieties (also known as (one-sided) ladder de- 
terminantal varieties). See I.KMY05..I for a treatment of the related algebraic geometry. 



5.1. Vexillary permutations and flaggings of partitions. Identify a permutation n ^ Sn 
with the square array having blank boxes in all locations except at (i, 7r(i) ) for i = 1 , . . . , n, 
where we place dots. This defines the dot-matrix of n. We associate the diagram 

D(7t] = {(p,q) G{1,...,n}x {!,..., n}:7r(p) > qandTr^^lq) >p} 

to 7T. Pictorially, if we draw a "hook" consisting of lines going east and south from each 
dot, then D [n) consists of the squares not in the hook of any dot. 
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Example 5.1. Letn 
bined below: 



123456789 
871 629534 



Its dot-matrix and diagram are com- 
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In what follows, we will assume our permutations n are vexillary, also known as 2143- 
avoiding: there exist no indices 1 <a<b<c<d<n with 7r(b) < 7r(a) < 7r(d) < n{c). 
We need some facts about vexillary permutations; further details consistent with the ter- 
minology and notation used here may be found in |KMY05| and the references therein. 

Throughout we will identify a partition A with its Young diagram. There is a partition 
A associated to n: let the kth diagonal of A (those boxes {(i, k + i)}) have as many boxes 
as the kth diagonal of D(7r), for each k. Indeed, this sets up a natural bijection between 
the boxes of A and the boxes of D(7r), taking the jth box down in the kth diagonal to the 
jth box down in the kth diagonal. (The difference is that in D(7r) there may be spaces in 
between the boxes.) This bijection also defines a flagging n on the rows of A. Namely, 
rii E N+ equals the row of D(7r) containing the box corresponding to the rightmost box of 
the i^^ row of A. We will thus speak interchangeably about n and the pair (A, n). 

In Example l5.11 the permutation n is vexillary, A = (7, 6, 4, 3, 2], and n = (1 , 2, 4, 6, 7). 

We remark that ft need not be a weakly increasing sequence. For instance, if ff = 

2 7 4 5 8 ^3 6 then A = (5,3,2,2, 1] and n = (2,5,4,4,5). 

Call a set-valued tableau t with shape A n-flagged if the maximum (so, the last) entry 
in each row is bounded above by the corresponding entry of n. 

Extend the definition of the empty-face tableau in the obvious way: it is the union of all 
the n-flagged semistandard tableau on the shape A. Let this set-valued tableau be denoted 
by EA,ft. (Note that EA,fi(b) is an interval in the natural numbers N: the smallest entry is 
the row position of b G A while the largest entry is the position of the corresponding box 
of D[n], under the bijection between A and D(7t) described above.) This gives rise to a 
tableau complex A(A, ft) generalizing that described in Section 1.2: 

Corollary 5.2. For a partition A and a flagging ft associated to a vexillary permutation tt, A (A, ft) 
is a simplicial hall, and its interior faces are the flagged semistandard set-valued Young tableaux. 

5.2. Formulae for vexillary Grothendieck polynomials. For each permutation tt G 
there is a (double) Grothendieck polynomial 



67t(xi, 



±1 



ill 



.Xn.-yi, . . .yn) G Z[xf\ . 

of Lascoux and Schiitzenberger fLS82l|. The case that n is vexillary has been of specific 
interest; see IFu92. .KMO 1 . .KM Y05 1 . We give tableau formulae in this setting. 
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Let SVT[X, n) denote the semistandard set-valued tableaux of shape A and flagging n. 
Similarly, denote by SSYT{7^,fi) and LSVT{'\,fi) the corresponding set of semistandard 
and limit semistandard tableaux, respectively For a set-valued tableau t, let T(b) denote 
the set of entries in box b. 

Corollary 5.3. Let tt G S^he a vexillary permutation and (A, n) be the associated partition and 
flagging. Each of the following is a formula for the double Grothendieck polynomial C57t[x, y). 

reLSVT{\n) bsA ieT(b) i6EA,s(b)\T(b) 

L M)'^'-™nn('-.ir;i,.,) 

reSVT{X,n) beA ieT(b) 

L n n^i-^^Viiiw) n ""^yi+^w 

reSSYT(\n) bGA teT(b) igE;^,^ (t>)\NT (b) 

Here, jib) = c(b]— r(b) isthedifferenceof the column and row indices of the box b G A. Moreover, 
in the last formula, is the tableau obtained by adding to each box b all entries o/t[b) either 
smaller than the entry o/T(b), or larger than the entry of rib) but such that replacing T(b] with 
this larger number would not give a tableau in SVT{X, n). 

Proof Formally, the second formula in the corollary is obtained as follows. Compute the 
K-polynomial of A-^ a via Theorem 14.31 using A, b, and i here in place of X, x, and y there. 
Then, for each fixed b and i, substitute the expression xtij^.j^j for the variable t^^^i). 

It was shown in |KMY051 that the second formula equals the desired double Grothen- 
dieck polynomial. Therefore, applying the same substitution procedure to the results of 
Proposition 14.11 and Theorem 14.51 yields two more formulae for the same Grothendieck 
polynomial. These formulae are, respectively, the first and third formulae here. □ 

In the above proof we appealed to BKMYO S^ to confirm that our K-polynomials are in 
fact Grothendieck polynomials. Let us briefly sketch how this can be proved directly; we 
refer the reader to |KMY05| for terminology To each vexillary permutation n there is an 
accessible box of A. From this one can define two vexillary permutations Up and Uc- We 
obtain a safe vertex of the flagged Young tableau complex by removing the largest entry 
of EA.fi that appears in the accessible box. The resulting deletion and star subcomplexes 
are naturally isomorphic to (multicones over) the flagged Young tableau complexes for 
n-p and Uq respectively. The recursion from Proposition 14.71 is precisely Lascoux's transi- 
tion formula for vexillary Grothendieck polynomials BLaOli lLa 03l (after the substitution 
t(xh4ij) ^— > ^iTJt^j(b))- Thus, since both polynomials satisfy the same recursion (and initial 
conditions), they are equal. 

Specializations of these formulae are of interest. Suppose we set ijj = 1 for each j and 
replace xt with 1 — xt for each i. If we assume furthermore that n is Grassmannian, then we 
obtain Buch's formula |Bu02| for the single Grothendieck polynomial C5a(1 — x) |Bu02|. 
If instead we take the lowest degree terms of the polynomial, we obtain Wachs's formula 
for a flagged Schur polynomial |Wa85|. Making both of these specializations gives the 
classical tableau formula for an ordinary Schur polynomial. 

We remark that it is also possible to use similar methods to extend these results to give 
set-valued skew tableau formulae for "321 -avoiding permutations" (see Example l2.9|l . 
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